Thanks for stopping by
our bulletin board.
Please take just a moment to register so you can post your own questions
and reply to topics. It is free and takes only a minute to register. Just click on the register
link
Don't worry about it... lot's of people are 'tards. My ex-wife was a 'tard... she's a pilot now.
----------------
OK, for those of you who are still awake, and not outside enjoying the great weekend weather...
Confidence intervals tell us about the potential inaccuracy of our estimates.
All statistics are estimates.
When Barry says that he looked at the scores of a number of examiners, he notes that some scored the same data differently. So we want to know how much those scores are the same (average), and we use that as our estimate. We'd never think of using a single examiner to estimate the accuracy of the polygraph. We'd prefer to have a number of examiners to use as the basis for accuracy estimates.
Similarly, we'd never want to use a single sample as the basis for the estimate of a technique. We'd prefer to have a number of different samples, and use the average.
While doing that we also have to be concerned about how much our observed results differ, among examiners and among different samples of data. That is why standard deviations are important.
Because we know that 68% of all scores will be with in 1 st-dev of the average, and 95% of all scores will be withing 2 st-deviations (assuming data are normal), we can estimate the likelihood or calculate the normal for which most examiners will achieve, even though there will be differences. Scores greater than two standard deviations from the mean are called "not within normal limits," or "outside the normal range."
So, the ideal situtation is to have access to so much data that we have averages of our averages (distributions of sample distributions). That is how we make things like IQ tests - we don't let one sample be the average.
With standard deviations we can also estimate the Standard Error of the Mean. Which allows us to calculate a confidence interval or the range that any given sample is likely to occur within.
Standard Errors (SEM or SE), are calculated as the standard deviation divided by the square-root of the N. So, larger samples are better, because they have larger square roots to divide into the StDev, and will provide smaller SEM (and because larger samples tend to be more normal).
A Confidence Interval tells us the likelihood that we would see a score within a certain range at any given time. Or, how many times we would see the score in that range if we did the same experiment 100 times.
There will always be variability, even if we have the same examiner score the same sample on two different days. (unless the process is entirely mechanical and entirely automated). This is what we mean by test-retest reliability - how likely are we to see the same result if we do it again. Another kind of reliability is interrater (inter-scorer) reliability, which tell us about how likely we are to get the same result if we have two different examiners score a test or sample of tests.
Try an experiment.
Find a quarter and flip in 100 times, and write down your results. We know the accuracy of a coin toss is .5 (assuming a fair coin). Do you think you will have exactly 50 heads and 50 tails? Probably not. That is because its a single sample. What will be more informative is a whole distribution of samples.
So, take your coin and flip it 100 more times and write down the results. Now do that 998 more times for a total of 1000 samples of 100 coin flips. (whew).
Then average all the results. How many heads, on average, and how many tails, on average, for every sample of 100 coin flips?
(lunch)
OK, I'm done.
Here is what I have.
Heads = average of 50.103
Tails = average of 49.897
At most I had 67 head in a single sample of 100 coin flips. I also had 66 tails one time. The minimum that I observed was 33 heads, and 34 tails in any sample of 100 tosses.
But on average it is 50, or close to that.
So we calculate the StDev. = 5.161 for both
Then we divide that StDev by the square root of 1000, which is 31.623, and find a Standard Error (SE) = .163
Because the distribution of sample distributions can be expected to be normally distributed, we know that 95% of the time (if we did this lots and lots of times) our observed results would be within two standard errors of the expected population mean (which for a fair coin is .5 or 50/100.
So, 50 + (1.96*.163) is the upper limit of our 95% Confidence Interval (CI), and
50 - (1.96*.193) is the lower limit of the 95% CI.
that gives
95% CI = 50.32 to 49.68
Because our results are 50.103 and 49.897, our observed results are within the normally expected range.
What we have proved is that we have a "fair coin."
Now, if we didn't want to all the trouble to flip a coin some 1,000,000 times (and if our kids balk too much), then we could have a computer do this, using Micro$soft Excel or the OpenOffice.org spreadsheet.
We start by creating a sample space of 100 cases, and generate random numbers between zero and 1. For every case with a random number less than .5 we call that "tails," and for ever case that is >.5 that is heads. Then we have the spreadsheet count the number of heads and tails and store the results. We build at macro loop using VBA code, and set the loop to run 1000 times.
Bada-bing-bada-boom!
We have now created a very simple Monte Carlo simulation of a coin flip experiment.
posted 01-26-2008 06:47 PM
This is similar to the material that I am sudying for the GRE. This is how the food chain works of examiners: I read Stats posts and think that I have never worked with anyone that writes like that.
I almost find it amusing when Stat is confused by your posts, Nelson, because that shows how far down the chain some of us(me) lie.
Nelson, I told you there were more charts coming if you don't mind, but I never sent them. Can I have that fax again, I wanted you to look at some very strange cardio in the relevants, it doesn't look like moves to me but I would assume he must be. It can't be CM's in the relevants. I don't know.
[This message has been edited by Buster (edited 01-26-2008).]
posted 01-26-2008 07:48 PM
Ray, thank you for taking the time to explain to me and others what the heck Monte Carlo calculations are. I think that I get it now. Some of the vocabulary will come with time, but your coin toss brought it home. While my clip of the day was helpful, it still left me thinking, "well, statistics are better comprehended and left to professional statisticians." Light bulbs are going off as to just what the hell you have been up to the last year with all of the computer calculations. You are using a computer(s) to view or formulate a "best scoring practices" mode---among other things? I knew you were plugging in charts and results, but I suppose I didn't understand the significance of the work. I guess I have been missing a sort of tarded official abstract from your work.
cool
I beg your pardon Buster, but everyone knows I am a moron when it comes to math. My username "stat" came about as I had never been on a web forum, so I assumed that I would have to repeatedly type my username-----so as a result of repeatedly typeing the word "stated" as in "the offender stated....", I chose "stat" as the letters are very close together and I can type "stat" in matter of 1/4 of a second. It didn't occur to me til much later that folks might think I fancied myself a "statistics fan." Like I said, I'm kind of dumb....ask my wife.
[This message has been edited by stat (edited 01-26-2008).]
Stat writes well, and I feel like I learn something daily from him and others. I also always look forward to learning from the well-informed information that Barry continually provides, along with the measured wisdom of skip and others.
Fax me any charts at
720 554 7677
faxing is good because it comes to me as a .pdf and I can screenshot those to .jpg fairly easily.
You can also email a PF, to
raymond.nelson@gmail.com
Looking at charts is more fun than looking at math.
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
Your right, some of this is better left to the professional statisticians - which I am not.
I'm just an average field polygraph examiner from Colorado.
I do have an interesting combination of cross-training, in field polygraph (PCSOT), psychology and psychometrics/psychodiagnostics and some experience in statistics. All of that makes it possible for me to push some of this right now, and its been gratifying to help try and solve some problems and try to make some sense of things.
Our field is at risk for falling behind other sciences. We should absolutely be interested in the most modern and advanced methods of studying data. At present, I probably couldn't pick up a journal in most fields of science without reading about some monte-carlo, bootstrapping, jack-knifing, or permutation experiment. Now a lot more of us know these words, and 2 or 3 people somewhere might become interested and may know what to do next.
I believe is is absolutely possible to find some incremental improvements in our methods. They'll never be perfect. Nothing will. But neglecting to study our work and study our data, is simply negligent. Standards and training are great, but they don't really move us forward much. Knowledge from data does move us forward. If we want to know where to go next, we must look at the data.
The real challenges are incremental:
What contributes to so many inconclusives?
How do we improve interrater reliability and reduce disagreement?
Is there anything about our decision rules that could improve our specificity figures to be more consistent with sensitivity estimates?
How can we begin to calculate the significance of hand scores and spot score, and can that contribute to better decision accuracy?
What can we do to reduce the inclusion of noise in scored data?
How can we better differentiate CMs from non-CM noise?
How do we move closer towards an integrative understanding of the various psychological constructs that account for and complicate polygraph reactions.
How do we better describe our polygraph data in language that accurately reflects what we know about physiology?
Paying attention to these types of questions will begin to make some of our detractors rethink themselves and will silence some naysayers. This is not just a job for lab-rats. All of us contribute, by simply noticing the conversations, in between the motorcycle madness and photoshop fodder.
I'm out - got an hour drive home.
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
Not to hijack your post, but a couple of things about these charts that have frustrated me for some time.
Two separate thefts totalling 10,000 dollars of jewelry. I have one of the sons of the victim. I get DI, but the software is all over the place.
DI- on polyscore Inc- on Nate Gordons Inc- on OSS.
This scares me because my score and Nates software almost always agree.
In post test interview he said the R's made him scared because everyone thinks he did it and thats all he can think about. I was really happy with setting up the controls and was happy on the pretest "reactions" for the controls.
He was adamant he didn't take them. The reason I am sending them is some strange cardio patterns. Also do you get that "everyone thinks I did" it excuse alot to explain reactions, because it also makes sense to me if it could overpower, even good, controls. I usually get the "well I did take money out of his wallet 5 years ago and I keep thinking about it" or "I thought about it so thats why I failed that question."
C5 Besides the bubble tape, UPTAO 20 DYE steal anything of value? R6 DY take that jewelry?
C8 IYEL DYE tell a serious lie? R9 DY take that jewelry?
C11 UPTAO 20 DYE steal from someone who trusted you? R12 Can you take me to that missing jewelry?
[This message has been edited by Buster (edited 01-27-2008).]
Yeah, this kid fits the profile. College kid studying computers, with credit card debt. Smoked weed the night before, had a smirnoff for breakfast the day of the test.
I sound like a broken record with this line, but the kid should have been an easy confession. He handled himself well post test. First words out of his mouth was calmly - "Well guess that thing's not accurate." To me his fear was obviously facing his family not the charges. I tried to alleviate those fears, but was unsuccessful.
I tested the brother and he was inconclusive leaning to truth. I wasn't real happy with those charts but I don't beleive he was involved. He has no debt and doesn't use drugs.
Detectives played this perfect. They interviewed him, a month passed due to some major B+E's, then called them and scheduled the test.
I told them my opinion, this kid was involved the other brother wasn't. But w/o the confession, in my opinion I didn't get the job done.
The detectives investigated and don't feel the contractors are involved.
posted 01-27-2008 09:52 PM
Yeah that basic math got me!
10-4 on the early EDA reaction, I could have hit an irrelevant there instead. Also I see the PVC there too. During the test, I don't remember seeing that, I have to be more perceptive to that.
A couple of those cardios, one is cutoff by the page, are gigantic.
This must have been a pain in the ass to transfer to WWW, so again thanx and thanx for the advice, I could use it.
I'm not even gunna touch your statistical explanations. You lost me at, "This is dry...
Buster, I was wondering why you are not using a movement sensor? Given the vast available propaganda and instruction, I would never consider running or grading any test without one.
posted 01-28-2008 12:26 PM
Sackett is right, we need to all be getting motion sensors. But, if for a while you cannot afford an instrument "upgrayedd"----and you have exhausted every effort to finance or lease one from an instrument company, than perhaps drastic measures are in order. If your instrument came with a plethysmographic finger sensor, a thumb sensor---any other kind of sensor or even an extra pneumograph (non-bellows type)---you need to strap it to the chair and leave it there before your examinee enters the rm, and when you explain the components before attachment of the body pieces, inform him that you have a state of the art sensor attched to his seat, so sensetive it can detect a mouse fart. If he has read TLBTLD and knows to look for cushion sensors, he might (might is optimistic) be disuaded from butt yawning. It needs to be actually hooked up to the sensor box, with the channel turned off, as examinees will eyeball your gear.
This emergency manuever of course is a lie. It is comparable to those phony video cameras at some retail stores. As you know, stores with real cameras still get robbed.
Many many examiners (especially cops with per diems and state-paid instruments) will rail against this idea of deceiving examinees about the component---my advice here. Or maybe not. Get the new instrument ASAP, but in the meantime, be resourceful.
[This message has been edited by stat (edited 01-28-2008).]
posted 01-28-2008 03:15 PM
....of course if you didn't want to stop with the phony cushion, you could buy a broken camera ($3 at flea market) that looks something like this with a homemade sticker attached;"Infrared"
....and then warn your examinee that you have an infrared camera that you view----and even make a comment as to according to readings at a glance, he appears to be thirsty ---tell him not to chew gum or you will see thermal traces off the charts----show him not the gear, but have this photo tacked to the wall next to his chair;
I know, I know, I am a devious bastard with all sorts of bad advice.
[This message has been edited by stat (edited 01-28-2008).]
you're tooooo funny! "Mouse farts?" "Butt yawning?", Thermal pics on walls? I believe you may have lost it....either that, or you miss the good ole' days of the zeroxolygraph machine!
posted 01-28-2008 03:40 PM
I have to give in to the cushion. I actually had a chance before Christmas when I was asked if I needed anything, but I chose to ask to be sent to an interrogation class instead.
I still am stubborn where I don't think the people I test are capable of countermeasures. Especially as I noted before, after we all tried them in the academy and they don't work.
I will submiss, I know its required by 2012. Thanx for the input.
Two random questions so I don't have to start a new thread.
1. Do you leave the room (Reid) after the test, or stay mark it in front of him and never leave the room before you start interrogation?
2. Do you combine all of your questions during pre test review, in Specific Issue Testing or do you introduce them seperately I, R, C? Thanx.
Yeah, Stat lost it, but we all benefit from it! Just kidding Stat.
[This message has been edited by Buster (edited 01-28-2008).]
posted 01-28-2008 04:06 PM
Buster, a little advice----OK, the pure and good kind this time.
Close and lock the exam room door.
Attach the components to yourself while viewing the screen.
Bite the S___ out of a tiny piece of your tongue---not in a wimpy way, but like it is your ass if you don't cause real pain. Do so while trying to control your breathing---it isn't perfect, but then again no pneumo data is. See that mountainous GSR that just happened about 3-4 seconds after the pain? How many recenterings did the auto do, 3,4 5? See the less dramatic but very visible cardio bump? Check out the pneumos. Yea, scary.
Now, do some "butt yawns"----1 at a time. Do them without moving your legs or making noise. Check out that cardio---did it climb like your little DI guy from charts posted on this thread----an inch? 3 inches? You also get a modest yet noticeable bump in the GSR. Check out the pneumos. Nothing except maybe some data noise, right?
And lastly, sit still--get the tracings to settle down. Now picture an axe tumbling toward your face. Not such a big arousal, but then try a physical countermeasure with that mental one after viewing a mental one by itself. It's not so hard that a redneck with a high speed modem can't learn if he were desperate.
Every examiner owes it to their profession to view these arousals. Remember, those fake "bio-feedback" arousals don't have a boost from psyche set like a field spot does.
[This message has been edited by stat (edited 01-28-2008).]
[This message has been edited by stat (edited 01-28-2008).]
[This message has been edited by stat (edited 01-28-2008).]
posted 01-28-2008 08:34 PM
Yup Stat, I have done stuff like that. A PA Trooper told me to attach myself and test myself. He said to go over speils and look for reactions. When I told another examiner that he said he never heard of it. The trooper said he attaches himself alot. I only did it a couple of times-- then I got lazy. Is like going to Anti- I just got lazy and don't go there anymore.
I just think that I have to "get real" and stop thinking that these people aren't smart enough to try that stuff on me.
I always assume PE testing and PCSOT testing would be full of CM's, but not my SIP's. I know I'm wrong. I'm just wet behind the ears and never wanted to face that stuff yet. I guess that was an excuse?
posted 01-28-2008 10:32 PM
Eric is right. I have done almost everything while attached to the polygraph (now E get you thoughts out of the gutter and I don't want to see any photoshops). One time I had my stereo on mute with the button ready under my finger to turn it on at HIGH volume - just to see my reaction. I have eaten beans - just to see my reaction w/and w/o the butt pad. I have bitten my cheek, tongue, pushed on the roof of my mouth, put a tack under my toe, pushed on a sore spot on my foot, had exciting thoughts, did mental math...... It is very wise to try everything and then touch the key board to indicate you are going to do something and then periodically review the reactions or redo/re-experience the CM's.
You just never know who will attempt to utilize CM's or just intentionally screw up your charts!
As for AP, don't get lazy, its our profession that is being targeted. Just don't become obsessed and addicted like a few of us....lol
[This message has been edited by Taylor (edited 01-28-2008).]
posted 01-28-2008 11:02 PM
Donna, I am going to defer to Ted regarding the open-ended remark of your polygraph experimentation. HE might be able to whoop you. Me, I'm not so sure---I certainly can't outrun a bullet.
Buster: I had a dumb, dull-affect (emotionless) high school kid give me brilliant countermeasures. If I posted them here and didn't tell you they were cm's I bet $100 no one could guess them. Uneventful pneumos, well-timed galvo, crescendo'd cardio (gradual). He admitted the countermeasures during our bible meeting not because I asked, but because he mistook a beginning post-test statement I made which was something to the effect of "these charts certainly tell a story."
I nearly sharted myself when he confessed--------------like Taylor during one of her NOW infamous "polygraph experiments." lol
-------------------------------------------- " When I die I want to go peacefully like my grandfather, not yelling and screaming like the passengers in his car...."
[This message has been edited by stat (edited 01-28-2008).]
posted 01-28-2008 11:24 PM
Eric, I was a range master for several years...lol
I would love to see those charts. What did he admit to doing?
BTW, fartin and shartin are two different things and I never sharted....and don't tell me you haven't tried it.
I think we need a new thread - every time I see statistics - part of my brain shuts down. A good thread could be BEST ONE LINERS TO GET AN ADMISSION. 1. These charts certainly tell a story. 2. Your in a deep hole and digging yourself deeper with all your lies, I am trowing you the rope, your life line, grab ahold of it.... (btw, you gotta keep a straight face with that one - but it really works). 3. Well I forgot what I said today to the sex offender after he failed but before he got the dry heaves, turned white, kept doing vomit burps and then made some major admissions...guess I should review the tapes, oh yeah, I said LIAR.
OSS-3 = DI. You can see the Cardio on Chart 3 was not scored because there is only one CQ measurement. The default configuration for OSS-3 is to not use a single CQ. You can replace the missing values if you want, but that will give an inconclusive result due the size of that first CQ cardio reaction.
OSS-2 = INC. OSS-2 is not robust against missing data (cumulative data model), and was trained with replacement data. So, the default configuration for OSS-2 is to fill missing RQ values with mean values, and to use the nearest extant CQ if necessary. The result is that this OSS-2 tool will score the cardio on Chart 3.
r
------------------ "Gentlemen, you can't fight in here. This is the war room." --(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
I am a lover-not a fighter! I too have spent countless hours attached to the instrument attempting to create different reactions. It is a healthy experiment. Cleve Backster became well known for this very thing. I however, do not plan to publish a study on "shartin during a polygraph".
Polygraph Place Bulletin Board
posted 01-26-2008 01:00 PM
posted 01-28-2008 03:36 PM
